Dynamic Adjustment of Concurrent Neural Networks within Limited Power Thermal Constraints in Autonomous Driving

Autonomous driving vehicles need to perform many different tasks like lane detection, blind-spot detection, and cruise control. Vision perception systems in autonomous vehicles run many neural networks concurrently to perform these tasks and thus have heavy processing workloads. The processors in automotive systems have significant power and thermal constraints that are dynamically changing in various field conditions. Existing techniques adjust processor and memory frequencies to regulate workloads to meet the required constraints. When presented with tight constraints, these solutions reduce the hardware speed and consequently increase the inference time of all the neural networks. Slower neural network inference is a safety hazard because mission-critical tasks like collision prevention are performed less frequently. We define an index, QoP (Quality of Perception), to represent the quality of the system-level vision perception. QoP requires high accuracy and processing speed on mission-critical autonomous driving tasks. We present a structured framework to maximize the QoP of the vision perception system within different power constraints. Our framework dynamically reallocates computing resources to the vision tasks by adjusting hyper-parameters of neural networks in run-time. The proposed method improves the QoP by an order of magnitude, which is a significant benefit for the reliability and safety of autonomous driving. Our experiments show that, in case of 15% power budget decrease, the proposed framework successfully maintains normal vision operations with high QoP (only $\sim$2.5% loss) while conventional methods of operating frequency throttling fail to function due to massive QoP degradation ($\sim$95% loss).